The present invention relates to virtual reality control for planning and interacting with stage lighting design and production. More specifically, the present invention defines a virtual reality design system that facilitates interaction between a stage lighting designer and many aspects of the stage lighting event.
The production of a modern stage performance is a complex undertaking. This is partially due to the complicated interaction of various effects created by the combination of lighting and sound apparatus. The designer of these effects combines all of these effects into a cohesive whole which furthers the artistic aims of the performance.
A myriad of details must be considered and planned in this design process. Designers or technicians must envision and create appropriate lighting and sound effects. That requires planning of numerous details of the equipment. These details include the location and operation of the equipment, wiring and location of the equipment at the desired locations, assembly and disassembly, and moving of this equipment between locations.
The design of stage lighting effects require the selection and configuration of numerous lighting fixtures, each of which has many adjustable parameters. For example, each light fixture can be adjusted in many ways, including position adjustment, intensity, color, and beam size. These parameters must be set to a very high degree of accuracy in order to satisfy the requirements of the designer. The effect also needs to be timed to coincide with the proper time in the show based on the musical program.
Previous design processes have used disorganized techniques for planning the operation, e.g., on paper, or by trial and error using actual equipment on a stage. To a certain extent, the designers must rely upon their own ability to visualize and predict the effects of a given lighting design.
Some progress has been made to improve the stage lighting design process by the use of computer lighting simulation systems such as MAC ICON, available from Light and Sound Design, Limited of Birmingham, England, and WYSIWIG, available from Flying Pig Systems, Inc. Both of these systems calculate and simulate lighting effects based on the lighting parameters that are indicative of a lighting show.
These lighting simulation systems produce a two-dimensional simulation of lighting effects on a stage. However, such systems are limited by the lack of realism in the two-dimensional display. Further, the complex user interfaces of these systems has generally restricted their use to designers having experience with such systems. Moreover, the two dimensional display of the inherently three dimensional effect has made it difficult for the designer to visualize the end result.
Once designers complete the lighting (and/or sound) design process, they must resolve a number of other practical issues before the desired effects can be implemented. This requires selecting appropriate lighting devices and parameter settings, and the dynamics and sequence of the desired effects.
The designer/technician must also configure electrical and structural systems to support and interconnect all of the lighting and sound apparatus in the appropriate location. The structural components include, for example, trusses and rigging, chains and chain motors which are used to raise the trusses and rigging. Also, each lighting fixture must be connected to cables which act as the source of power and control. Therefore, each show requires a routing plan to locate these cables in an efficient path from the fixtures to a source of power and/or control.
The lighting design also needs to account for the unique requirements of each stage. Conventionally, a drawing is used to model the stage to determine where the rigging, trusses and cables will be placed. This drawing is then used to determine where all of the cables will be routed, including the lighting fixtures, dimmer racks, consoles, or other locations. It is now common to use this drawing to locate the structural elements, and to decide what lengths of cables are required, by actually measuring the lengths on the scale drawing. Then, numerous cables are cut and constructed to the desired lengths.
Other structural considerations include determining the location in the building where trusses can be hung. Different points in the roof and walls of different structures have different loading capabilities. The designer must consider the weight and forces of these load points on the structure to ensure that known capacities of the structure are not exceeded.
Problems can arise with the above-described design process. For example, creating drawings is very time consuming. Errors often occur in measurements made using these drawings. There may not be time to correct such errors given the severe time constraints in the construction of stage performances. This is particularly true in shows that travel frequently. Also, calculation of structural load values and limits is time consuming and prone to human error.
Moving the lighting and sound equipment presents another challenge for stage production. Crews assemble, disassemble, pack, and transport lighting and sound equipment. Loading these items onto a truck requires careful consideration of many factors, which is usually done by a highly experienced road boss. For example, some fixtures ride better in certain parts of the truck. It may be desirable to locate heavier items, such as dimmer racks, towards the front of the truck. Once a certain amount of weight has been placed on one portion of the truck, additional heavy items should be placed on another part of the truck, or in a separate truck.
It is also important to consider the stacking ability of each item. This defines the kinds of items, and the maximum weight, that can be placed on top of a given item without causing damage. It is important to consider the size and arrangement of items to optimize usage of truck space. This minimizes the number and size of the trucks.
The sequence of packing and unpacking is also important. For example, since trusses support the entire lighting apparatus, packing them in the front of the truck necessitates unloading the entire truck before set up can begin. This dictates a preference for loading the trusses last so that they can be removed first. Similar considerations apply to rigging equipment, including motors, span sets and other devices used for handling trusses. There is no need to unload truss-hung lights until the trusses are unloaded. However, it is acceptable to unload floor mounted lights, at any time so they do not need to be packed in a specific location. Also, when multiple trucks are used, the road boss must consider which crews are packing and unpacking different areas at different times to avoid different crews interfering with each other.
Currently, all of these considerations enter into decisions regarding when and where to pack items in a truck. Performing this task often leads to inefficient truck packing, because of the difficulty in visualizing and planning for all of these packing considerations. Unfortunately, trial and error is slow and error prone. This can result in delays or damage to the equipment and/or truck.
Another problem is timing. Shows must be frequently put together and taken apart in order to allow them to arrive in the proper place at the proper time. Usually the road boss makes an estimate of how long the truck packing, driving, and unpacking will take, and tries to operate based on those estimates. However, this requires experience and is prone to errors. The actual operation could take much more or less time.
Another interesting feature noted by the inventors of the present invention was the possibility of simulation of a performance, using a karaoke style.
The inventors of the present invention have recognized all of the above needs arising from creating lighting and sound designs, and mounting, interconnecting and shipping the necessary equipment. They have realized that this is essentially a three-dimensional problem that is being solved using two dimensional tools.
The inventors recognized that the three dimensional user interface could facilitate entry of all of this data and information. The preferred user interface is a virtual reality operating system. This allows simulation of the three dimensional problem more accurately than by using two dimensional tools.
Virtual reality allows an individual to interact with a computer simulation in a three-dimensional virtual world using multi-dimensional and multi-sensory interactive interface devices. Because the human brain has a high degree of skill in recognizing spatial patterns, the complex relationships inherent in these design tasks are more readily apparent in a three dimensional visual-spatial display. Examples of such virtual reality devices include head mounted xe2x80x9ceye phonesxe2x80x9d providing a visual display of a simulated scene; tactile sensitive gloves worn by the user for touching, pointing, and interacting with objects in the simulated display; sound input to the user corresponding to sounds generated in the simulation; and a voice recognition system responsive to the user""s verbal commands. Hand and body motion sensors enhance the virtual reality experience by sensing turning of the user and then shifting the view in the visual display, just as the view would shift when turning in the real world. The combined result of these kinds of interfaces can generate a realistic sensation of actually existing in the virtual three dimensional environment. Virtual reality has been the subject of many patents and publications, including, for example, the book xe2x80x9cArtificial Realityxe2x80x9d, by Myron Kruegger, and U.S. Pat. No. 5,423,554, which is herein incorporated by reference.
In view of the above-discussed issues in stage production and design, this invention uses virtual reality interfaces to provide the designer with an improved media to allow planning of various aspects of the stage lighting show.
One planning scheme is the simulation of the three-dimensional stage lighting effects.
Techniques are also described to simulate the apparatus producing the effects. This provides the designer with an accurate and intuitive control over the simulation, as well as an accurate view of lighting displays, scenery and other equipment including sound. This will improve the quality and efficiency of the design process.
Another virtual reality tool simulates the process of packing and unpacking lighting and sound stage equipment into a truck. This allows the user to interact with an apparently three dimensional simulation of the actual packing and unpacking process. Since the packing is being simulated on a computer, this also allows the computer to keep track of various aspects of the packing including any rules that may be violated by the packing.
For lighting design, the user inputs data regarding the characteristics of the stage and light fixtures, including the dimensions and constraints of the stage and building, the dimensions of each light fixture, the types of light fixtures, the point of reference on the stage, the location of each light fixture, and the parameters of each light fixture.
The user also inputs the type, dimension, and weight of each light apparatus and support structure, and their cabling requirements. Further, the user inputs the constraints of the building and light apparatus support structures. For the task of packing the light and sound equipment, the user inputs data regarding the characteristics of the truck and the packages. These include the dimensions of the interior of the truck, the dimensions of each package, the center of gravity, and preferences and constraints for each package. Alternately, a database for various standard types can be prestored into the computer. This database can also include information about any warning conditions, such as maximum unbalanced load and others.
Another system used by the present invention enables determination of how long the truck packing which is simulated during the truck packing exercise will actually take.
Yet another system of the present invention relates to setup of the eventual devices. The setup parameters allow the lighting fixtures to be placed in various locations on the stage. Proper operation of those fixtures is obtained by running cabling links to those locations, and calculation of safety parameters associated therewith.
The data input devices can include a keyboard, a disk drive, and virtual reality (VR) devices, such as interactive gloves and head mounted visual display of the type usually used as virtual reality input devices. The processing subsystem stores and processes all of the data about the characteristics of the stage and light fixtures, along with structural and packing data. The system then simulates the lighting effects, the apparatus producing these effects, and the packing environment.
The processing subsystem has a processor, a memory, and processor overhead hardware. The processor runs a simulation module and a virtual reality operating system. The simulation module simulates lighting effects, the light and sound equipment, their supporting structures, as well as the truck and its packages. The virtual reality operating system allows the user to enjoy real-time control over the simulation; it also produces a three-dimensional image showing the simulated elements and lighting effects. The three-dimensional image is displayed on display devices, including a monitor and an interactive helmet of the type usually used as a virtual reality display device.
The above described input data is used to form a computer-based simulation. The user can interact with this simulation by reaching into the virtual image and adjusting its characteristics. For example, if the user is not satisfied with the lighting effects on the stage, the user reaches into the virtual image of the stage and grabs one or more of the light fixtures. A drop down menu preferably appears, displaying characteristics of the light fixtures. The user can easily adjust a parameter of the light fixture by simply pointing to the parameter on the drop down menu. Alternatively, the user may use voice commands to modify the simulation. In the lighting design process, the user can adjust the position of the light fixture by simply grabbing the light fixture and pointing it in the desired direction. Another simulation displays the ICON(trademark) control console which is normally used to control the lighting system, and allows the user to enter commands in VR space on the simulated console.
In the packing process, the user can reach into the virtual image of the truck and grab a package. The user can then place the package in the truck. If the package fits in the truck at the desired location, the virtual reality operating system produces a three-dimensional image showing the package in the truck at the selected location. Also, the user is notified if any of the input activities violate any of the design constraints placed on the system.
The location of devices can also be done in the VR space, by placing particular trusses/lamps in the desired location. The computer system prestores loading characteristics of the supports. When a load is placed in a loaded location, its weights and weight distribution are calculated to determine if any weight limitations have been exceeded.
The cabling and its routing are also calculated from the entered information, and diagrams facilitating the cabling and its layout can be prepared and used.
Yet another system enabled according to the present invention is an entirely new system of simulation. Previous simulation systems have been known, based on the Japanese xe2x80x9ckaraokexe2x80x9d style. These systems allow a user to sing or play along with musical accompaniment, and essentially become part of that musical accompaniment.
The present invention for the first time uses virtual reality to form a karaoke style simulation system. Preferably this is carried out by forming a simulation of, for example, a rock and roll band with or without lighting effects, playing the actual music. The user places himself in the virtual reality environment, essentially seeing himself in the presence of the rock and roll band and the stage lighting effects. This xe2x80x9cultimate karaokexe2x80x9d provides the ability to add and or remove aspects of the simulation, and to provide a more realistic simulation for the user.